Roadmap

• Fixing ideas: the possibility of empirically grounded process tracing
• Some bad news: limited gains from details (with Dawid, Musio, \(\downarrow\) )
• Some good news: moderators can provide foundations for qualitative tests
• Some bad news: nothing from nothing (\(\downarrow\) with Jacobs)
• Some good news: clues in many places
• Some OK news: applications

Roadmap

• Focus here on causes of effects (“explanation”)
• Mechanisms as clues for explanation

Same structures hold for mechanistic queries and non-mechanistic clues

The estimand

The estimand is:

• Is \(Y\) due to \(X\)? or
• Would \(Y\) have been different if \(X\) had been different in this case?

We often seek the probability of causation. Using potential outcomes notations: \(\Pr(Y(0)=0 | Y(1)=1)\).

Arguably this is an answer not an estimand. Still it’s our focus: we seek the answer that is defensible given the data and discuss the identifiability of this quantity.

Intuition 1

Say that we have lots of data from a randomized experiment and we know that the effect of X on Y is 2/3. In particular we have infinite data supporting the following conditional distribution of \(Y\) given an application of \(X\):

What is the probability that \(X\) caused \(Y\) for a case from this population? (an “exchangeable” case)

Intuition 1

From this data alone either of the following distributions of potential outcomes are possible:

• Positive effects for 2/3 of units; negative for 1/3
• 0 effect for 2/3 of units, positive for 1/3

PC is 1 in the former case and 0.5 in the latter case so bounds are [0.5, 1]

Intuition 1

Sometimes distributions allow for tighter distributions:

Here \(X=1\) is necessary for \(Y=1\). From this we know that if \(X=1, Y=1\) then \(X=1\) caused \(Y=1\)

But for a \(X=0, Y=0\) we don’t know if \(X=0\) caused \(Y=0\)

Intuition 1

Sometimes distributions allow for tighter distributions:

Here \(X=1\) is sufficient for \(Y=1\). From this we know that if \(X=0, Y=0\) then \(X=0\) caused \(Y=0\)

But for a \(X=1, Y=1\) we don’t know if \(X=1\) caused \(Y=1\). In fact POC = \(0.5\).

Intuition 2

Say now that we could decompose the \(X\rightarrow Y\) process into a 2 step process. \(X\rightarrow M \rightarrow Y\).

For an \(X=1, Y=1\) case, is learning about \(M\) informative for the probability that \(X=1\) caused \(Y=1\)?

Intuition 2

Imagine:

1. \(X\) is sufficient for \(M\) and \(M\) is necessary for \(Y\)
2. \(X\) is necessary for \(M\) and \(M\) is sufficient for \(Y\)

We learn nothing in the first case but might learn a lot in the second case.

Intuition 2

Take the second case. We compose this:

with POC of \(\left[\frac13, \frac23\right]\). Into:

• POC: \(1 \times 0.5 = 0.5\) if \(M=1\); and 0 if \(M=0\).
• expected value \(1/3\) (since \(\Pr(M=1|X=Y=1) = \frac23\)).

Intuition 2

Say now that we could decompose the \(X\rightarrow Y\) process into a 10 step process, with an effect of 0.9 at every step.

Then the upper bound at each step remains at 1. The lower bound is \(\frac{0.9}{0.95}\) which is about 95%.

However \(0.95^{10} < 0.60\) which means bounds are now [0.6, 1]: so better but not much better than what we had before.

Intuition 2

Knowing a lot about many steps means that you have greater certainty at each step, but there are more sites for leakage and so the accumulation of confidence is not large.

Homogeneous transitions

Suppose all matrices are equal:

PC bounds (red for observed, blue for non observed mediators) tighten only modestly as the length of the homogeneous chain increases

Non-homogeneous transitions

We find the largest and smallest upper and lower bounds from any complete mediation process, for different types of evidence.

• all achievable by decompositions of length one or two

Comparison with other bounds

• compare these results with possible bounds from monotonicity, and from knowledge of covariates.

Take homes

• POC in general not identified
• Knowledge of mediation processes can help; information on mediators can help further
• Positive evidence on mediators is of quite limited value for inferring causes of effects
• The best possible gains are achievable from short chains: long chains leak
• Negative evidence can however be powerful

Clues in many places

The general idea for case level process tracing following data based model training can be generalized to:

• arbitrary DAGs
• arbitrary queries
• finite and incomplete data

See Humphreys and Jacobs Integrated Inferences

Strategy

• Embed prior causal beliefs explicitly in a causal model
  • What might directly cause what?
  • Where might there be confounding?
  • How likely are different causal effects?
• Possible observations are nodes in the model
• Probative value of observations emerges from properties of the model
• Then gather data and update the model
• Yields inferences from data that are consistent with explicit prior beliefs about how the world works

Generality of the approach

• Basic structures described by Pearl (2009) yield conceptual basis/language to articulate possibly complex estimands
• Flexible stan structure used for estimation

Clues in many places

• Mediators
• Moderators
• Confounds
• Colliders
• Surrogates
• Symptoms

Clues in many places

Simulate performance

We imagine a real world in which there are in fact monotonic effects and no confounding, though this is not known. (The data suggests a process in which \(X\) is necessary for \(M\) and \(M\) sufficient for \(Y\))

Define the model, then update, and query.

Hoop illustration implications

We see that we can find \(M\) informative about whether \(X\) caused \(Y\) in a case specifically when we see \(M=0\).

This is striking because:

• We have a non experimental design in which the effect of \(X\) on \(Y\) is not identified
• The probability that \(Y\) is caused by \(X\) is also not identified.
• We placed no restrictions on functional forms except those implied by the DAG (notably \(X\) works via \(M\))
• We did not build “hoopiness” into the model, we learned about it from the model

We can do similarly for other Van Evera tests, specifically using moderators to generate “doubly decisive” tests.

Nothing from nothing

• Say you had access to large amounts of observational data on \(X\), \(Y\) and \(M\)
• You know the temporal order of \(X, Y, M\) only.
• Can you figure out whether \(M\) is informative for \(X\) causes \(Y\) from this data?

Updating gives:

Assume a world, like above, where in fact \(X \rightarrow M \rightarrow Y\), all effects strong (80%, 80%).

This negative result holds even if we can exclude \(X \rightarrow Y\)

This example illustrates the Cartwright idea of no causes in, no causes out.

Institutions and growth

Institutions and Growth Model

Updated

Case level inferences given possible observations of distance and mortality.

Implications

• for a case with weak institutions and low growth (first column), the former likely caused the latter. Similarly for cases with strong institutions and growth (last column).

• cases with weak institutions and high growth (and vice versa) relationship unlikely causal

• in a strong institutions / high growth case, proximity to the equator increases confidence that the strong institutions helped: despite the fact that distance and institutions are complements for the average treatment effect

• Mortality is informative about the effect of institutions on growth even if we already know

• Learned patterns of confounding are consistent with a world in which settlers responded to low mortality by building strong institutions specifically in those places where they rationally expected strong institutions to help.

Confounding

Correlated posteriors

Warning

Complexity grows quickly

Causes of effects

We consider a binary treatment variable \(X\) and binary outcome variable \(Y\). We have experimental data supplying values for \(\Pr(Y=y \mid X\leftarrow x),\) where \(X\leftarrow x\) denotes a regime in which \(X\) is set to value \(x\) by external intervention. Define \[\begin{eqnarray*} \tau &:=& \Pr(Y=1\mid X\leftarrow 1) - \Pr(Y=1\mid X\leftarrow 0)\\ \rho &:=& \Pr(Y=1\mid X\leftarrow 1) - \Pr(Y=0\mid X\leftarrow 0). \end{eqnarray*}\] Then \(\tau\) is the average causal effect of \(X\) on \(Y\), while \(\rho\) is a measure of how common the outcome is.

Transition matrix

The transition matrix from \(X\) to \(Y\) is then \[\begin{equation} \label{eq:P} P = P(\tau,\rho) :=\left( \begin{array}[c]{cc} \frac 1 2(1+\tau-\rho) & \frac 1 2(1-\tau+\rho)\\ \frac 1 2(1-\tau-\rho) & \frac 1 2(1+\tau+\rho) \end{array} \right) \end{equation}\] where we have \[\begin{equation} \label{eq:rhotau} |\rho| + |\tau| \leq 1. \end{equation}\]

We have equality if and only if one of the entries of \(P\) is 0, in which case we term \(P\) degenerate.

Potential responses

We have observed treatment \(X=1\) and outcome \(Y=1\), and want to know the probability that the treatment caused the response. We introduce the potential responses, \({\mbox{$\mathbf Y$}}= (Y_0,Y_1)\), with \(Y_x\) the potential value of \(Y\) when setting \(X \leftarrow x\). We will assume no confounding ( \(X \hspace{1.5mm}\perp\hspace{-3.9mm}\perp\hspace{1.5mm}{\mbox{$\mathbf Y$}}\)). \

The probability of causation is defined as:

\[\text{PC}: = \Pr(Y_0 =0 \mid Y_1 =1, X=1)\]

Joint distribution of \(Y\)

The joint distribution for \({\mbox{$\mathbf Y$}}\) has the form of the following table:

The entries of the table are not determined by \(P\), but have one degree of freedom, expressed by the ``slack’’ quantity \(\xi\) = \(\xi(P)\).

We only know \(|\tau| \leq \xi \leq 1-|\rho|.\) In particular if \(\tau>0\), \(X=1\), \(Y=1\), we have}

\[\begin{equation*} \text{PC} = \frac{\xi+\tau}{2\Pr(Y=1 \mid X \leftarrow 1)}, \end{equation*}\]

with the following interval bounds:

\[\begin{equation*} \text{LB}:=\frac{2\tau}{1+\tau+\rho} \leq \text{PC} \leq \frac{1+\tau-|\rho|}{1+\tau+\rho} =: \text{UB}. \end{equation*}\]

\(\text{PC}\) is identified if and only if \(|\rho| = 1-\tau\), which holds when \(P\) is degenerate with either the lower left or upper right element of \(P\) being 0. In the former case \(\text{PC}=\tau\), while in the latter case \(\text{PC} = 1\). More generally, we have \(\text{LB} = \tau/\Pr(Y=1 \mid X \leftarrow 1) \geq \tau\), so \(\text{PC} \geq \tau\).

Mediation

We can sometimes improve these bounds if we can observe additional variables.

Here we consider a chain of binary mediators that form a complete mediation sequence \[X \equiv M_0 \rightarrow M_1 \ldots \rightarrow M_n \equiv Y\]

• Suppose we are able to determine \(\Pr(M_{i+1} = m_{i+1} \mid M_i \leftarrow m_i)\) and that there is no confounding at every step

• In this case we have a (generally non-stationary) Markov chain.

• Let the transition matrix from \(M_{i-1}\) to \(M_{i}\) be \(P_i = P(\tau_i,\rho_i)\), and the overall transition matrix from \(X\) to \(Y\) be \(P = P(\tau,\rho)\).

• We have \[P = P^{(n)} :=\prod_{i = 1}^n P_{i}, \quad\quad \tau := \prod_{i=1}^{n} \tau_i,\qquad\qquad \rho := \sum_{i=1}^{n} \rho_i\prod_{j=i+1}^n\tau_j.\]

Chain inference

We have: The average causal effect of \(X\) on \(Y\) is the product of the successive average causal effects of each variable in the sequence on the following one.

Chain inference from clues

We introduce the potential variables \({\mbox{$\mathbf M$}}_i :=(M_{i0}, M_{i1})\), where \(M_{im}\) denotes the potential value of \(M_i\) under \(M_{i-1} \leftarrow m\) (supposed unaffected by values of previous \(M\)’s).

We impose mutual independence between \(X\), \({\mbox{$\mathbf M$}}_1\),,\({\mbox{$\mathbf M$}}_n\) (no confounding).

• We show: Knowledge of the mediation process does not change the lower bound for \(\xi\). However, the upper bound is typically reduced.